FCoE Linux Management Tools Installation Guide

The FCoE Linux Management Tools included in this package are:

fcoeadm - program to create, reset, destroy, and display FCoE interfaces

fcoemon - program to monitor the events from the DCB daemon

Requirements:

The HBAAPI library and the HBAAPI vendor library source must be build and installed before you can build the management tools. The HBAAPI vendor library, libhbalinux, may be downloaded from www.Open-FCoE.org. The instructions in the package describes how to download and build the libraries. See the man pages for other requirements.


fcoeadm

The \fBfcoeadm\fR command is intended to be the FCoE management tool for the Linux systems. The \fB\-c\fR, \fB\-d\fR, and \fB\-r\fR options are used to create, destroy, and reset an FCoE instance on a given network interface. The other options are used to query the information of the FCoE instance which includes the interface information, target information, LUN information, and port statistics. The \fBfcoeadm\fR command invokes the \fBHBAAPI\fR library routines to obtain these information. The \fBHBAAPI\fR library routines invoke the vendor\-specific library, \fBlibhbalinux\fR, to grab the information from the /sys file system. In other words, the \fBfcoeadm\fR command requires to have \fBlibHBAAPI\fR and \fBlibhbalinux\fR installed on the system to work.

The \fBlibhbalinux\fR is maintained at \fB\fIwww.Open\-FCoE.org\fR. The installation instructions of \fBlibhbalinux\fR also instructs how to download the \fBHBAAPI\fR source code, build and install with the \fBlibhbalinux\fR. The last option \fB\-h\fR is used to show a brief usage message of the supported command syntax.

Options:

\fB\-c\fR, \fB\-\-create\fR \fI<ethX>\fR

Creates an FCoE instance based on the given \fI<ethX>\fR.

\fB\-d\fR, \fB\-\-destroy\fR \fI<ethX>\fR

Destroys an FCoE instance based on the given \fI<ethX>\fR.

\fB\-r\fR, \fB\-\-reset\fR \fI<ethX>\fR

Resets the fc_host associated with the FCoE interface given by \fI<ethX>\fR.

\fB\-i\fR, \fB\-\-interface\fR \fI<ethX>\fR

Show the information of the FCoE instances created at \fI<ethX>\fR. If \fI<ethX>\fR is not specified the command will show the information of all the FCoE instances created on the system.

\fB\-t\fR, \fB\-\-target\fR \fI<ethX>\fR

Show the information of all the discovered targets from the FCoE instances created at \fI<ethX>\fR. If \fI<ethX>\fR is not specified the command will show the information of all the discovered targets from all the FCoE instances created.

\fB\-l\fR, \fB\-\-lun\fR \fI<target_port_id>\fR \fI<lun_id>\fR

Show the detailed information of a specific LUN with \fI<lun_id>\fR at the target with port id \fI<target_port_id>\fR. port id is also known as FC\-ID. If \fI<lun_id>\fR is not specified, all the LUNs associated with the target will be shown.

\fB\-s\fR, \fB\-\-stats\fR \fI<ethX>\fR \fB\-n\fR \fI<interval>\fR

Show the statistics (including FC4 statistics) of the FCoE instances created at \fI<ethX>\fR. The information will be display in one line on the screen per given time interval. \fI<interval>\fR should be specified in whole intergers greater than 0. It specifies the time interval in the unit of second. If \fI<interval>\fR is not specified, the default interval is one second.

\fB\-v\fR, \fB\-\-version\fR

Displays the version of the \fBfcoeadm\fR command.

\fB\-h\fR, \fB\-\-help\fR

Displays the usage message of the \fBfcoeadm\fR command where \fI<ethX>\fR is the network interface name, such as eth0, eth1, etc.

Examples:

Creates an FCoE instance on eth2

$ \fBfcoeadm\fR \-c eth2

Destroys the FCoE instance on eth2

$ \fBfcoeadm\fR \-d eth2

Resets the FCoE instance on eth2

$ \fBfcoeadm\fR \-r eth2

Show the information of all the adapters and their ports having FCoE instances created.

$ \fBfcoeadm\fR \-i

Show the information of a specific interface eth3. If eth3 has no FCoE instances created, the command will show the error "No fc_host found for eth3".

$ \fBfcoeadm\fR \-i eth3

Show the information of all the discovered targets from all the ports having FCoE instances created (they may be on different adapter cards). A brief listing of discovered LUNs are listed after the target they are associated with, if any.

$ \fBfcoeadm\fR \-t

Show the information of all the discovered targets from a given port (eth3) having FCoE instance created. A brief listing of discovered LUNs are listed after each target they are associated with, if any.

$ \fBfcoeadm\fR \-t eth3

Show the detailed information of all the LUNs associated with a specific target. The target is identified by its port id (aka FC\-ID) 0xD700EF.

$ \fBfcoeadm\fR \-l 0xD700EF

$ \fBfcoeadm\fR \-l D700EF
$ \fBfcoeadm\fR \-l 0xd700ef
$ \fBfcoeadm\fR \-l d700ef

Show the detailed information of a LUN associated with a specific target. The target is identified by its port id (aka FC\-ID) 0xD700EF and the LUN is identified by its LUN id.

$ \fBfcoeadm\fR \-l 0xD700EF 1

Show the statistics information of a specific port eth3 having FCoE instances created. The statistics are displayed one line per time interval. The default interval is one second if \-n option is not specified.

$ \fBfcoeadm\fR \-s eth3

$ \fBfcoeadm\fR \-s eth3 \-n 3
$ \fBfcoeadm\fR \-s eth3 \-n3

Reporting Bugs:

If you have identified a defect please either file a bug or engage the development mailing list at <http://www.Open-FCoE.org>.

Support:

Open\-FCoE is maintained at <http://www.Open-FCoE.org>. There are resources available for both developers and users at that site.

fcoemon

The \fBfcoemon\fR command is a FCoE management tool provided by the Open\-FCoE package.

\fBfcoemon\fR is the daemon of the \fBfcoe\fR system service. When \fBfcoemon\fR starts, it establishes a socket connection with the \fBDCB\fR daemon. It then sends commands to, and receives responses and event from the \fBDCB\fR daemon. The \fBfcoemon\fR will process the received responses and events and invoke the \fBfcoeplumb\fR script to create and destroy the FCoE interfaces.

Since \fBfcoemon\fR depends on the existence of \fBDCB service\fR, there are settings required for \fBDCB\fR before \fBfcoemon\fR can be started. See the \fIDCB Settings\fR section below.

Notice that the fcoe system service does not depend on dcbd service. However, the fcoemon daemon will be started by the fcoe service only if any one of the Ethernet port requires DCB service. In this case, fcoe service depends on dcbd service. If none of the Ethernet ports requires DCB service, the fcoemon will not be started and, in this case, the fcoe service does not depend on dcbd service.

Options:

\fB\-h | \-v | \-\-version\fR

Show the version of the \fBfcoemon\fR command.

\fB\-f | \-\-foreground\fR

Run \fBfcoemon\fR in the foreground.

\fB\-d | \-\-debug\fR

Enable debugging messages.

\fB\-e | \-\-exec\fR \fI<path\-of\-fcoeplumb\-script>\fR

Specify the location of the \fBfcoeplumb\fR script.Terminology:

\fBPFC\fR \- The DCB Priority Flow Control feature.

\fBApp:FCoE\fR \- The DCB Fibre Channel over Ethernet feature.

\fBLLINK\fR \- The DCB Logical Link TLV (or Logical Link) feature.

\fBmultiq\fR \- See Documentation/networking/multiqueue.txt of linux kernel 2.6.28 or higher.

\fBskbedit\fR \- See Documentation/networking/multiqueue.txt of linux kernel 2.6.28 or higher.

Installation Requirements:

The DCB and FCoE kernel configuration options must be enabled, these were introduced in v2.6.29. Both the linux kernel and iproute2 must support multiq and skbedit. The DCB must be installed with version 0.9.4 and higher.

Supported DCB Events:

In response to each supported event from the DCB daemon, the fcoemon collects the current settings from the DCB daemon and decide whether to delete and re-add the multiq queue discipline and skbedit filter. The \fBfcoemon\fR does not destroy, reset, or create FCoE interfaces during the DCB event processing.

\fBFEATURE_APP\fR

If an event message is received from dcbd and if the feature code in the event message is FEATURE_APP (5), and if the subtype field is APP_FCOE_STYPE (0), we got a mode or configuration change event of the FCoE application. The fcoemon will then issue queries to the DCB daemon to collect the current mode and configuration information.

\fBFEATURE_PFC\fR

If an event message is received from dcbd and if the feature code in the event message is FEATURE_PFC (3), we got a mode or configuration change event of the Priority Flow Control (PFC) feature. The fcoemon will then issue queries to the DCB daemon to collect the current mode and configuration information.

\fBFEATURE_LLINK\fR

If an event message is received from dcbd and if the feature code in the event message is FEATURE_LLINK (6), and if the subtype field is LLINK_FCOE_STYPE (0), we got a mode or configuration change event of the Logical Link TLV feature. The fcoemon will then issue queries to the DCB daemon to collect the current mode and configuration information.

Criteria of Creating, Resetting and Destroying FCoE Interface

In this section the \fBdcbtool\fR is used to describe the conditions of the DCB feture status because the meaning is more understandable and precise. Although you may also issue the commands at run-time, the commands are intended only to be used for description purpose.
 
\fBPFC and App:FCoE\fR

DCB is configured correctly if

0
3
"1)" " The command \fBdcbtool gc ethX dcb\fR shows DCB State: on"
3
"2)" " The command \fBdcbtool gc ethX app:0\fR shows Enable:true,"
3
" " " Advertise:true, Willing:true."
3
"3)" " The command \fBdcbtool go ethX app:0\fR shows OperMode:true."
3
"4)" " The command \fBdcbtool go ethX pfc\fR shows OperMode:true and"
3
" " " the values of pfcup."
3
"5)" " The command \fBdcbtool go ethX app:0\fR shows appcfg. The bits"
3
" " " set to 1 are also set to 1 in pfcup found in (4)."

\fBLogical Link TLV\fR

The Logical Link TLV feature is configured correctly if

0
3
"1)" " The command \fBdcbtool gc ethX ll:0\fR shows"
3
" " " Enable:true, Advertise:true, Willing:true."
3
"2)" " The command \fBdcbtool go ethX ll:0\fR shows OperMode:true."
3
"3)" " The command \fBdbtool gp ethX ll:0\fR shows Link Status:up."

\fBCriteria to create FCoE interface\fR

If DCB is required at the Ethernet port, a FCoE interface may be created only if the DCB and the Logical Link TLV feature are configured correctly. If DCB is not required at the Ethernet port, the FCoE interface may be created. FCoE interfaces are normally created by the \fBfcoe\fR system service.

\fBCriteria to Destroy FCoE Interface\fR

An FCoE interface will only be destroyed when the \fBfcoe\fR system service is stopped.

\fBCriteria to reset a FCoE interface\fR

The \fBfcoe\fR system service does not reset any FCoE interfaces.

\fBChanging DCB Configuration, Qdisc and Filters\fR

Changing the DCB configuration, qdisc, and filter are considered to be administrative actions.

When the \fBfcoe\fR system service starts up, it sets up the default DCB configuration, qdisc, and filter for reliable FCoE operations. Administrators may alter the configuration while the service is running.

Changing the DCB parameters may cause the fcoemon daemon to delete the existing multiq queue discipline, skbedit filter and re-add; but the fcoe service will not touch (e.g. destroy or reset) the FCoE interface.

Changing the DCB configuration, qdisc, and filter should be avoided while I/O traffic are in progress.

Files:

The Installation of the Open-FCoE management tools include the following files:
\fB/etc/fcoe/config\fR

This is the common configuration file for the \fBfcoe\fR system service. This file will be read by the \fI/etc/init.d/fcoe\fR" script, the \fI/etc/fcoe/scripts/fcoeplumb\fR script, and the \fIfcoemon\fR daemon. The default options in this file are:

\fBDEBUG="yes"\fR and \fBUSE_SYSLOG="yes"\fR.

The former is used to enable (select yes) or disable (select no) debugging messages of fcoemon, the \fIfcoe\fR service script, and the fcoeplumb script. The latter is to indicate if the log messages of fcoemon, the \fIfcoe\fR service script, and the fcoeplumb script are to be output to the system log. Use editor to set the desired \fByes/no\fR values.

\fB/etc/fcoe/cfg-ethX\fR

There is one of this file for each Ethernet interface \fBethX\fR found in the output of

\fBcat /proc/net/dev\fR at the time of installation. This file will be read by the \fI/etc/init.d/fcoe\fR script and the \fIfcoemon\fR daemon. The default options in this file are:

\fBFCOE_ENABLE="no"\fR and \fBDCB_REQUIRED="yes"\fR. The former is used to enable (select yes) or disable (select no) the FCoE service at the ethX port. The latter is to indicate if the DCB service is required (select yes) or not required (select no) at the ethX port. If the former is set to no, the latter is ignored. The selection of the settings should match the settings of the FCoE switch port connected to the local Ethernet ethX port. Use editor to set the desired \fByes/no\fR values for the \fBethX\fR interfaces.

\fB/etc/init.d/fcoe\fR

This is the \fIfcoe\fR system service shell script. This script is invoked by the \fBinit\fR process or by the \fBservice\fR command.

\fB/sbin/fcoemon\fR

This is the \fIfcoemon\fR daemon only invoked by the \fIfcoe\fR system service script.

\fB/sbin/fcoeadm\fR

This is the program used by the \fIfcoe\fR system service to create or destroy FCoE interfaces.

\fB/etc/fcoe/scripts/fcoeplumb\fR

This is a script only used by the \fBfcoemon\fR daemon.

Reporting Bugs:

If you have identified a defect please either file a bug or engage the development mailing list at <http://www.Open-FCoE.org>.

Support

Open-FCoE is maintained at <http://www.Open-FCoE.org>. There are resources available for both developers and users at that site.

Last modified on 3/31/10 4:19p Revision 6